Skip to content

Add reform impacts dashboard planning doc (#806)#1732

Open
vahid-ahmadi wants to merge 1 commit into
mainfrom
vahid-ahmadi/issue-806-reform-impacts-dashboard-plan
Open

Add reform impacts dashboard planning doc (#806)#1732
vahid-ahmadi wants to merge 1 commit into
mainfrom
vahid-ahmadi/issue-806-reform-impacts-dashboard-plan

Conversation

@vahid-ahmadi
Copy link
Copy Markdown
Collaborator

Summary

  • Add dashboard tracking reform impacts #806 asks for a dashboard tracking high-importance reform budgetary impacts over time and across model / data versions.
  • Key audit finding: the fixtures already exist. tests/microsimulation/reforms_config.yaml has 9 canonical regression entries (basic-rate +1pp, UC taper to 20%, VAT +2pp, etc.) with expected impacts and tolerances, and test_parametric_reform_impacts.py exercises them on every PR. What's missing is a time series view onto them.
  • New planning page at docs/book/validation/reform-impacts-dashboard-plan.md proposes:
    • append-only storage of (timestamp, version, dataset, reform_name, expected_impact, observed_impact, gap) rows on a CSV / Parquet (HuggingFace dataset or gh-pages artefact);
    • scheduled runner (weekly + on tag) walking reforms_config.yaml;
    • three publication paths in order of effort — docs page Plotly chart, living JSON endpoint, policyengine.org dashboard tile;
    • annotations for each known regime shift (EFO release, dataset refresh, methodology PR);
    • initial reform list = the existing 9 fixtures plus a roadmap for v2 (two-child limit removal, PA freeze extension, employer NI changes).
  • Cross-links to the nowcasting comparison, RF methodology, and child-poverty validation pages so the four together function as a coherent "how is the model doing" view.
  • Pure planning; no code or fixture changes.

Test plan

  • Markdown lints cleanly (cross-refs, MyST admonition).
  • Docs build job picks up the new page under validation/.

🤖 Generated with Claude Code

#806 asks for a dashboard tracking high-importance reform budgetary
impacts over time and across model / data versions.

Auditing the existing infrastructure:

- reforms_config.yaml already has 9 canonical regression fixtures
  (basic rate +1pp, UC taper to 20%, VAT +2pp, etc.) with expected
  impacts and tolerances.
- test_parametric_reform_impacts.py runs each fixture in CI and
  fails on drift outside the tolerance.

So the *fixtures* exist; what's missing is a published *time series*
view onto them.

The page proposes:

- append-only CSV / Parquet storage of (timestamp,
  policyengine_uk_version, policyengine_uk_data_version, dataset_url,
  reform_name, expected_impact, observed_impact, gap),
- a scheduled GitHub Actions runner (weekly + on tag) walking
  reforms_config.yaml,
- three publication paths in order of effort: docs page Plotly chart,
  living JSON endpoint, policyengine.org dashboard tile,
- annotations for EFO releases, dataset refreshes, methodology PRs.

Cross-links to the nowcasting comparison and child poverty validation
pages so the three together form a coherent "how is the model doing"
view.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant